Adaptive machine and its thermodynamic costs 
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We study the minimal thermodynamically consistent model for an adaptive machine that transfers 
particles from a higher chemical potential reservoir to a lower one. This model describes essentials of 
the inhomogeneous catalysis. It is supposed to function with the maximal current under uncertain 
chemical potentials: if they change, the machine tunes its own structure fitting it to the maximal 
current under new conditions. This adaptation is possible under two limitations, i) The degree 
of freedom that controls the machine's structure has to have a stored energy (described via a 
negative temperature). The origin of this result is traced back to the Le Chatelier principle, ii) 
The machine has to malfunction at a constant environment due to structural fluctuations, whose 
relative magnitude is controlled solely by the stored energy. We argue that several features of the 
adaptive machine are similar to those of living organisms (energy storage, aging). 

PACS numbers: 05.65.+b,05.10.Gg, 05.20.-y 



I. INTRODUCTION 

Adaptation is one of the paradigms of biology and com- 
plex systems theory, but its investigations [1-5] rarely 
start from the first principles of thermal physics (instead, 
they proceed with mathematical [3] or qualitative ap- 
proaches [4, 5]). Hence not much is known about the 
physical costs of adaptation. Besides its fundamental 
importance, this question is relevant due to increasing 
interest in smart (self-controling) materials [6] and due 
to miniaturization of technologies that make the external 
control impossible or obsolete. 

Consider a machine that transports matter with the 
maximal current allowed by external constraints. This 
optimal functioning will be seen to demand a good fit 
between its structure and external environment (making 
the machine somewhat similar to an organism). For ex- 
ploiting such a machine in an uncertain environment one 
can control it externally, or design a specific direct inter- 
action between the environment and the structure. Here 
we explore the most interesting possibility: upon envi- 
ronmental changes, the machine tunes its own structure 
so as to work optimally under new environment. Such 
a machine is adaptive without external control. Ordi- 
nary macroscopic machines are not adaptive in this sense: 
their structure is either predetermined or is controlled ex- 
ternally. This is why the laws of thermodynamics focus 
on the impossibility of achieving certain tasks via exter- 
nal fields without feedback [7] ^. 



^ Feedback control, where the action of external fields takes into 
account some information on the system's state, is also a tradi- 
tional subject of thermodynamics [8—11]. However, in the major- 
ity of papers on this subject people are interested by the usage 
of feedback in extracting more work or in reducing the entropy 
[10, 11]; two original intentions of the Maxwell's demon [8] (see, 
however, [9]). Moreover, the feedback is typically described ex- 
ternally, i.e. without making the controller an integral part of the 



But small machines are able to alter their own struc- 
ture. In certain enzymes and ion channels the functional 
part (performing the catalysis) couples to the conforma- 
tional part that in its turn back-reacts on the functional 
part [13, 14]. Structure-function interaction exists also in 
inhomogeneous catalysis [15]. It modifies the catalyst's 
structure and changes the catalytic current. 

Our purpose is to understand thermodynamic limits of 
adaptation via the minimal model of a structure-adaptive 
machine transporting particles from one reservoir to an- 
other. We choose this model for three reasons. First, it 
realizes the simplest and most fundamental machine-like 
function (catching and releasing); hence its understand- 
ing can infiuence the design of future adaptive machines. 
Second, the model adequately describes the essentials of 
inhomogeneous catalysis. Third, this is a step towards 
studying more complex systems of biochemical catalysis 
which evolved to increase their current [1, 2]. Our per- 
sonal motivation of studying adaptive machines is a belief 
that these non-living systems may demonstrate certain 
key features of living organisms. 

Here is a brief description of the adaptive machine to 
be elaborated below. For an environment with given 
chemical potentials, our model machine transports parti- 
cles from the higher chemical potential to the lower one 
and does so with the maximal current (or speed) once 
its structure (energies of its states) fits that particular 
environment. Upon changing the chemical potentials the 
functional part tunes the structure so that the machine 
functions with the maximal current under new condi- 
tions. We study thermodynamical costs of this adapta- 
tion. (We focus on changes of chemical potentials, since 
the chemical potential difference is the driving force of 



described set-up. In contrast, we are interested here by the feed- 
back processes that maintain a maximal current on the face of 
environmental changes, and we describe the controller explicitly. 




in our study). The stationary (time- independent) prob- 
abilities pf of Fi (i = 1,2) have the grand-canonical 
(equilibrium) Gibbsian form [16, 17] ^ 



p[cq]^g-B.+piV,^ 



(1) 



The relaxation of F towards its equilibrium state (1) can 
be described by the master equation [17]: 



Api/At = V .[PjjPj - PjiPt] 



«,i = l,2, (2) 



FIG. 1: The global scheme of the model. Two-level system F 
transfers particles from a reservoir with chemical potential /il 
to that with jUr. The controller S tunes the energy difference 
A_E of F and interacts with a bath at temperature 1/6. 



the particle transport.) 

This work is organized as follows. The next two sec- 
tions define the main ingredients of the model. Section 
IV defines the concept of adaptation, as applied to our 
situation. Sections IV and V identify the main thermo- 
dynamic costs of adaptation. Section VI discusses certain 
alternative set-ups of adaptation, e.g. when the control- 
ling degree of freedom is allowed to sense directly the 
uncertain environment. The last section summarizes our 
work. Here we also discuss how the thermodynamic costs 
of adaptation relate to basic characteristics of aging and 
energy storage known for living organisms. The paper 
has five appendices. 



II. THE MODEL: FUNCTIONAL DEGREE OF 
FREEDOM 

Our model has two degrees of freedom: functional and 
structural; see Fig. 1. They couple to each other forming 
together an autonomous system. First we shall discuss 
the dynamics of the functional degree of freedom F as- 
suming that the structural degree of freedom is fixed. 



where pi {pi + p2 = 1) is the probability of F,; and pi^j 



is the transition rate F,- 



F,-. Since the bath is in 



equilibrium, pi^j hold the detailed balance condition: 



/\E = E2 ~^ E\ . 



e^^-V2i, 



(3) 
(4) 



Eq. (3) ensures that the stationary state of (2) coincides 
with (1). 

Without loss of generality we parametrize transition 
rates pi2 and p2i as 



T 



T 



(5) 



where t is the time-scale induced by the interaction with 
the reservoir. In (2), r scales the running time (and hence 
the relaxation time), but does not appear in the equi- 
librium probabilities (1). Microscopic derivations of the 
master equation show that r depends on the features of 
the reservoir (e.g. its energy spectrum), but can also de- 
pend on the internal parameter AQ = AE — p; see [18] 
and (25) below. Since AQ is the heat received or trans- 
ferred to the reservoir, the time-scale t has the global 
minimum at AQ = 0: it takes longer to transfer a larger 
amount of heat. This holds for all physical cases we are 
aware of. 



A. Definition of F 

F is a simple model for a trap (or adsorption center). 
It has two states F^ (i = 1, 2): an empty state Fi whose 
energy is Ei and a filled state F2 with one particle and 
energy E2- Hence each state has energy Ei and carries 
Ni particles(i = 1,2): N2 = I and Ni = 0; see Fig. 2. 
Without coupling to external reservoir(s), F will stay in- 
definitely in one of its states, since the energy and the 
particle number are conserved. Hence any change be- 
tween the states of F is driven externally. 

Let us first assume that F interacts with an equilib- 
rium reservoir at chemical potential p and temperature 
T — 1 (this value of temperature is chosen convention- 
ally, since temperature gradients will not play any role 



^ In this paper we study distinguishable classical particles, while 
discrete energy states have the usual (e.g. in chemical physics) 
meaning of deep minima of potential energy. But we stress 
that the probability p^ oc e"'^'"*"^^'"^ for a n-state sys- 
tem (n may be infinite) interacting with an equilibrium reser- 
voir at temperature T and chemical potential fi is the general 
expression for the grand-canonical equilibrium [16]. Here each 
state i has energy Ei and the particle number Ni. For instance, 
this description applies to indistinguishable Bose or Fermi par- 
ticles [16]. We do not consider these cases in the present arti- 
cle, but for illustrative purposes let us remind how this formula 
applies to non-interacting Fermi particles; each particle has en- 
ergy levels £1, ...,eMj and not more than one particle can be in 
the same energy level. Now the states i can be parametrized 
via the filling numbers nj. = 0, 1 (A: = 1, ...,M) so that nj. is 
the number of Fermi particles having the energy e^. Now Ei 



EfcLi"fcandpl<=<il{nfe}: 



E{nk} = Efeii efeife, Ni = N{nt} 

g-(B{n,}+MiV{n,,})/T/^^ ^here Z = rifciiCl + e-f^*.-^)/^) is 

the statistical sum for Fermi particles. 






B. Master equation for two reservoirs 

In the equilibrium state (1) all currents nullify; this 
is the main message of the equilibrium state and it is 
ensured by the detailed balance condition (3) [17]. 

We are interested by transport due to a chemical po- 
tential gradient. Hence we assume that F simultaneously 
couples with two equilibrium reservoirs (L and R) of en- 
ergy and particles ^. Their temperatures are equal, 



Tt = Tr = T = 1, 



but the chemical potentials are different 



HL > m- 



(6) 



(7) 



F will transport particles from L to R; see Fig. 2 and (15) 
below. Its dynamics is described by a master equation 
(2), but now once F couples simultaneously with L and 
R, 



Pij — Pij\L + Pij\R, 



(8) 



where Pij\h and Pij\Yi are the transition rates driven by 
separate reservoirs [22] . Since L and R are in equilibrium, 
Pijlk satisfy detailed balance [cf. (3)] 



Pl2|k 






L,R. 



(9) 



As (2) shows, F relaxes in time from any initial proba- 
bility to the stationary (but generally non-equilibrium) 
probability 



Prj/iPt] + P]i) 



(10) 



If L and R are in mutual equilibrium (p, — fxi^ — p^), 
Pi reverts via (9) to the equilibrium (Gibbs) probability 
(1). We can apply the same parametrization as in (5) 



Pl2\k 



- i_p^[AB-MkAW] 



Tk 



P21\k 



1 
— ( 

Tk 



-AE+ii^AN] /j^j^N 



where Tk is the time-scale of F — k interaction (k = L, R). 
The discussion given after (5) now applies to Tk in sepa- 
rate. In particular, Tk — as a function of AkQ — A.E — ^]^ 
(heat received or transferred to the reservoir k) — has the 
global minimum at AQk — 0. 

In contrast to the equilibrium situation, now the time- 
scales Tk will generally appear also in the stationary prob- 
ability (10); see (18) below. 



^ A somewhat more realistic assumptions would be that F consists 
of two interacting two-level systems FI-*^! and F^l so that FI^I 
(FI^I) couples only with L (R). If the interaction between FI^I 
and FPI is very strong — they are forced to be simultaneously in 
their up or down states — we can effectively replace F^^l and FI^I 
by a single two level system that couples simultaneously with two 
thermal baths; the states, where F^^l is up while FI^I is down (or 
vice versa) have too large energies to be populated. 




FIG. 2: Two states of the functional degree of freedom F; 
see section II A and II B. The state Fi (F2) is empty (filled 
with one particle) and has energy Ei (-E2). The particle in F2 
can move to one of the reservoirs (L or R) causing transition 
F2 — )> Fi. The reverse transition Fi — > F2 is due to catch- 
ing a particle from one of the reservoirs. The mechanism of 
transporting particles from L to R is that a particle comes 
from L and then jumps to R; see (15). 



C. Particle current 

1. General definition 

Our main target is the particle current: the mean num- 
ber of particles entering to F per time-unit from the k- 
reservoir. To find the current, let us note that the time- 
derivative of the average number of particles 



dt 



Y^N^p, - T.,N,h - E,.J^ 



k=L,R 



(12) 



-^k ^^.N^[p,J^^^p■j - Pj.ikP,], k = L,R, (13) 

is a sum Jl + Jr. of two separate contributions. Jk is 
due to interaction with the reservoir k, i.e. it is formed 
by the transition rates Pij\k coming from the reservoir k. 
These transitions rates satisfy the detailed balance with 
respect to the reservoir k; see (8, 9). Since the particle 
number is conserved during the interactio of F with each 
reservoir, the contribution Jk to -rx'^iNiPi coming from 
the reservoir k is identified with the current of particles 
from the reservoir k [11, 12] ''. Then (12) expresses the 
conservation law ^ . 



^ This widely applied identification of the particle current be- 
comes completely explicit within microscopically derived models 
of master-equations; see [18, 19] for reviews. 

"■ If there is only one reservoir acting on F, the particle current from 
it can be measured simply via the time-derivative -rfZli^iPi °f 
the average number of particles. This, of course, already implies 
that the number of particles is conserved. If there are two or 
more reservoir acting on F, the current of particles coming from 
a speific reservoir has to be measured via the reservoir. 



Eqs. (2, 13) lead to 



D. Maximization of the particle current 



Jk = P21\kPl- Pl2\kP2, k = L,R, 



(14) 



where we recalled that A^2 = 1 and A^i = 0. It should 
be intuitively clear from (14) that J^ is indeed the av- 
erage number of particles entering from the reservoir k: 
P2i\kPi is (proportional to) the probability for F to make 
transition Fi -^ F2 induced by the reservoir k (hence F 
catches a particle from the reservoir k). From this one 
subtracts the probability Pi2\kP2 of the reverse event: the 
particle leaving from F to the reservoir k. 



2. Stationary situation 

In the stationary (time-independent) situation Jl -I- 
Jy^ — A J2^ Nipi — 0; see (12, 13). As expected, there is 
only one independent current in the stationary situation. 
Using (10, 14) we get 



We want to have the largest Jl for given /iL and fx^., be- 
cause this means the optimal functioning of the matter- 
transporting machine. We start with a general premise 
that the current Jl should be finite. This implies from 
(16) two natural constraints [T is a constant, k=L,R]: 



Tk>r, 



(19) 



i.e. time-scales cannot be too short. The largest Jl for 
given /XL, Hk and (19) is obtained when we maximize (16) 
with all the three involved parameters tl, tr, AE being 
independent under conditions (19): 



Jl = max [JlJ = -= smh[ J 

Ti5,AB / 4 



which is reached for the optimal values 



(20) 



= r^ - r. 



AS* = /2 = (/iL + m)/2. (21) 



Jl = 



P21|LP12|R ~ P12\LP21\R 
P12 + P2I 



(15) 



Eq. (15) has a transparent interpretation that explains 
the mechanism of the particle transport from L to R; see 
also Fig. 2 in this context. Indeed, p2i|L is (proportional 
to) the probability that F transits Fi ^- F2 under influ- 
ence of L. This means that the particle in F came from 
L. Likewise, pi2\K is the probability that the particle will 
leave to R. Hence P2i\lPi2\r - Pi2\lP21\r in (15) is (pro- 
portional to) the probability that the particle came from 
L and leaves to R minus the probability of the reverse 
sequence of events. 

Eqs. (2-14) imply for the stationary values 



sinh[iii^] 



TLCOsh[^^^] 



•rRCOsh[^^] 



(16) 



Pi 



1 + e 



-AE+l- 



_i"L+ilR 



P2 



I -Pi, (17) 



; = ln (rL+TRe^^)/(TR, + TLe^^) , (18) 



where AE is defined in (4). 

Now Jl > for /t^ ^ Pr' the stationary current is 
from the higher chemical potential to the lower one. 

Note from (17) that provided that / ~ 0, the sta- 
tionary probability pi of F has a Gibbsian form with 
the temperature T — 1 and chemical potential iitiiis.; 
Z = is achieved under /^l ~ Pr (overall equilibrium) or 
Tl = Tr. If the latter condition holds, ^'""^^"- is in be- 
tween of the chemical potentials of the reservoirs L and 
R; hence F is generally not in equilibrium with them, 
even when it has a Gibbsian form. The fact of having a 
Gibbsian form — that physically means the existence of a 
local equilibrium — will be seen below to have important 
consequences. 



Eq. (20) is got as follows. We maximize Jl over tl and tr 
and obtain the first condition in (21). Next we maximize 
over AE. 

Here are the implications of (20-21). 

1. The optimal TjJ are fixed by the constraints. In 
contrast, AE* depends on the environment (reservoirs). 
The optimality condition tl = tr implies I = from 
(18). Hence the stationary probabilities pi for F^ have the 
Gibbsian form with chemical potential (21); see (17) and 
the discussion after (18). This consequence of the current 
maximization is one the main causes of our results below. 

2. The machine performing optimally in one environ- 
ment will be sub-optimal in another. Indeed, let the pa- 
rameters be fixed at their optimal values (21), and the 
chemical potentials are slowly changed as 



PL 



Pl, PR 



Pr, 



p'l > Pr, P 



p' = p + S,{22) 



where 6 = ^{p{^ - Ph + Mr - Pr)- 

The stationary current in the new situation (22) is 



Jl[Pl,Pr] ^JI[Pl,Pr]/cosH^], 



(23) 



where J£ [/ij^ , Mr] is the optimal current in the new envi- 
ronment; see (21, 16). For \6\ :^ 1 we get Jl[p'l,p'r] ^ 
Ji,[Pi^, p'ji]- the current becomes suboptimal for any siz- 
able environmental change. If the chemical potential dif- 
ference is conserved, /t^ — /Jr ~ pi^ — Pr, the current will 
decrease from its old value: Jl[Ml'Mr] ^ Ji\pl, Pr\- 

3. The formal reason of this fragility is that the op- 
timal AE* depends on the environment. Its physical 
reason is that F has to perform equally well two comple- 
mentary things: to bind and release. To illustrate this 
point, recall that N2 = 1, Ni = Q and assume in (21, 11): 



TL = Tr = T= 1. 



(24) 



Then the binding transition Fi — > F2 is driven mainly by 
the L reservoir, p2i|L > P21|R: while the releasing tran- 
sition F2 — > Fi is driven mainly by R: pi2\n > Pi2\l 
(this is why F transports particles from L to R). In the 
optimal regime both P21|L ^nd jOi2|r should be possibly 
large. Hence they are equal: /021|l ~ Pi2\r = e^^^L-Ma)^ 
i.e. F binds and releases the particle equally well. Now 
after the environmental change (22): P21|l/pi2|r = e^^^- 
Thus for 5 > {5 < 0) the current is sub-optimal, since 
F binds the particle better (worse) than releases it. 

4. Our model relates to inhomogeneous catalysis. Any 
catalysis facilitates the spontaneous transfer of reacting 
molecules from a higher to a lower chemical potential 
[20]. During an inhomogeneous catalysis the reactants 
are bound strongly to an active center of the catalysing 
surface, so that the reaction can proceed. But the re- 
action products should be weakly bound to the center, 
so that they are easily released making the center ready 
for a new reaction. This complementarity between bind- 
ing and releasing is essential for any good catalyst; e.g. 
silver and tungsten are both not good catalysis for or- 
ganic molecules: silver binds reactants too weakly, while 
tungsten binds the products too strongly [20]. 

Another situation, where binding and releasing are si- 
multaneously important is the oxygen transport by mam- 
mal erythrocytes. They are periodically removed from 
the blood, since they cease to perform well one of these 
functions, e.g., they bind oxygen too strongly [21]. 

5. Recall that in deriving (20) we assumed that the 
current Jl can be maximized over independent parame- 
ters tl, tr, AE; see also 2, where this assumption was 
used implicitly. While this leads to the largest value of 
Jl — in the sense that any relation between the three pa- 
rameters can only reduce the optimal value (20) — it is 
still possible that Tk does depend on AQk ~ AE — /ik, 
e.g. in the activated transport [18] 



Tk = cxp[wk + -|A(3k| 



(25) 



where Wk is the barrier height. We now show that in the 
linear regime /iL ~ /^r we can recover the same conclu- 
sions as above without assuming that Tk does not depend 
on AQk- 

In the linear regime we put /zl — p-R — M everywhere 
besides sinh[i^i^^] 



i^±^ in (16): 



Jl 



Ml -Mr 



2[rL(AQ)+TR(Ag)]cosh[ 



A_E- 



(26) 



where tl and tr are functions of AQ ~ AE—jl [18] . They 
have global minima at AQ = [see our discussion after 
(5)]. In the linear regime the stationary probabilities of 
F are naturally Gibbsian, since Z = in (17, 18). 

Maximizing (26) over AE we get J£ = 2[r,w+rH(o)] 
reached for AE* ~ p. In the linear regime these agree 
with (20). Also, we reproduce the conclusions of 1-4 
by keeping the dependence of Tk on AQ. Note that the 



changes in (22) should respect the linear regime: /i^ — 

Mr ~ Ml - Mr ~ 0. 

6. We aimed to show that in the linear regime our con- 
clusions on the current optimality and its fragility apply 
more generally. We do not assume the linear regime for 
the rest of this paper. Below we set tl = tr = T, be- 
cause this maximizes Jl with all other parameters being 
fixed. For technical simplicity we from now on put T = 1 
[cf. (24)]. 



III. STRUCTURAL DEGREE OF FREEDOM 



A. Master equation 

We discussed in 2 that the environmental changes (22) 
diminish the optimal current. If our machine is supposed 
to work in such an uncertain environment, the only possi- 
bility to ensure its optimal functioning is to assume that 
its structure S changes and adjusts the energy difference 
to AE = fl' after each environmental change p, ^ p,'; 
see (22). Then the current is maximal under each en- 
vironment. To account for structural changes we thus 
introduce a controller degree of freedom S, with states 
{S"}^^]^; see Fig. 3. We shall demand that S is slower 
than F and that it does not couple directly to the chang- 
ing environment. 

Let pf and £"" be, respectively, the joint probability 
and energy of S"Fi. Since S does not couple to particle 
reservoirs L and R, each state S"Fi carries the number 
of particles iV^ that does not depend on a; see Fig. 3. 
Hence the S — F coupling goes only via the energies Ef". 

The dynamics of S is driven by a thermal bath at a 
temperature 1/0; see Fig. 1. It will be seen below that 
the adaptation makes necessary for the baths of F and 
S to be different. Hence we assume that the baths of S 
and F are independent, and the general master equation 
P? = EjylRfj'p] - R]tP?] for S + F reduces to 

A" = J2,lp?jP? ~ p%p?] + eE,K">7 - ^7>?], (27) 

where Lof^ and /?"■ = X]k=L RP?i\k ^^^ ^^^ rates of tran- 
sitions FiS'' -^ FiS" and F^S" -^ F^S", respectively, 
e is the ratio between the time-scales of F and S. The 
detailed balance for p"-M reads [cf. (11)] 

M?2|k = e^*^^°"''°"^-\ M2i|k = 1/M?2|k, (28) 

where in (28) we used the settings (24); recall 6 in section 
HD. 

The detailed balance condition for uif^ is written down 
by analogy to (11) 



^"7 ^ giBf^ + |(£7--E°) ^ao 



B? 



(29) 



where B"^ relates to the inverse time-scale, and 9 is the 
inverse temperature of the bath of S [recall (6) in this 
context] . 




FIG. 3: The joint states of functional degree F and the struc- 
tural degree S. For simplicity it is assumed that both F and 
S can have two separate states (Fi,F2 and S^,S^, respec- 
tively); recall Fig. 2. Hence there are four states F^S" (of 
F -I- S) with energies Ef . The transitions FiS" -> F^S'', 
FiS" — > FjS™ take place between states having one common 
index; see (27). This is due to the fact that the reservoirs 
that drive the transitions are independent from each other. 



B. Time-scale separation 

It is generally understood that control processes in bi- 
ology involve time-scale separations betvi^een controlling 
and functional degrees of freedom; see [25, 26] for re- 
views ^. In line with this, we assume that S is much 
slower than F for e — >■ 0. One introduces in (27) the 
conditional probability p^^ defined via Pi|Q,p" = pf, and 



notes that in p'^ 
and p>j|a = 0(e) 



P"Pi\a+P°'Pi\_a we havep"pi\a = C(l) 
[23]. Hence Pi I Q and p" decouple: 



Pi\a 



E 



JT^i 



PtjPjla " PjrPt\c 



(30) 



P =^ 



[l]"^'' - f]''>"], l]"-^ == V ujf^p.i^. (31) 



dynamics of S one can replace in (31): 



n" 



n"^ = J2 



^t p 



2|7- 



(32) 



Then (31) becomes a Markov master equation for S: the 
future of S is determined by its own present state only. 
F influences S indirectly via the transition rates 0,°''^ . 

Since most of the time F is in its stationary situations 
corresponding to a fixed S" , the functioning of the ma- 
chine is described by the current J^ i^ the states S" and 
the stationary probability p" of S" found from (31, 32). 
This is the main consequence of the time-scale separa- 
tion. 

Using (17) with / = [see (31)] we get from (32, 29): 



1 + 



J(E^,-E?) l + £ 



p.-AE° 



1 + e^ 



' + 1 ( AB*" - A.E" ) - A_E'' -l-p 



-A_E/3 



1 + e''°'' + f (AB°-A£;'3)_A£;°+Ai ' 



(33) 
(34) 



At this stage we need to specify the transition rates 
wf^, because on one hand we want to have a non- 
trivial structure of S (including the possibility of tak- 
ing X — > oo), while on the other hand we want to get 
p°' explicitly. We thus choose to work with the birth- 
death model for S [17]: the states {S°'}a=i with ener- 
gies {-Ef }^=i form a one-dimensional chain. In (27-31) 
we allow transitions only between neighbouring states: 
S" ^ S"+\ a = l,...,i^- 1. Then the stationary 
probability of S reads from (31, 32) 



^"=^'11^=1 (^''^''Vf^'''^^')' "-2,..., if, (35) 
where p^ is determined from X]a=i-P" ^ 1- 

IV. ADAPTATION 



Eq. (30) describes the evolution of F for short times when 
S is fixed in its state S". Then F relaxes to its con- 



Pl2 



-A_E° 



ditional equilibriumpi|Q — „ , „ — i^^ i ^ j , 

where AS" = E!^ - Ef; see (10, 17). Since this relax- 
ation happens faster than S changes, for describing the 



The reasons for a widespread applicability of the time-scale sep- 
aration are summarized as follows, (i) It allows to reduce the 
complexity of the overall problem by separating (on fast times) 
the involved degrees of freedom into statistical (functional) and 
mechanical (structural) [25, 26]. In particular, this means that 
the stability of the fast subsystem is determined under fixed val- 
ues of the slow degrees of fredom, a fact that we stress before 
(35). (ii) The evolution of slow degrees of freedom is robust with 
respect to those parameters of the fast subsystem that govern its 
dynamics, but do not show up explicitly in the (conditional) sta- 
tionary probabilities that determine the effective transition rates 
of the slow subsystem [25]; see (32) in this context. 



A. Definition of adaptation 

The intuitive notion of adaptation is that this is a 
change in the system compensating environmental ef- 
fects. The general definition of adaptation was attempted 
in literature several times and led to interesting discus- 
sions; see [4, 5] for reviews ^. But within general defini- 
tions some important aspects of adaptation are left open. 



^ It is sometimes said that the fact of adaptation depends on the 
level of description; see [5]. This statement can be illustrated 
via a relaxator, a system that relaxes to its final stationary state 
from a set of initial states. If the relaxator is perturbed to one 
of the initial states it relaxes back to the stationary state. The 
perturbation is viewed as an environmental change which is com- 
pensated when the relaxator goes back to the stationary state. If 
one stays at the phenomenological description of the relaxator. 



the main of them is why (let alone how) the system is go- 
ing to compensate environmental effects ^. 

In our situation the adaptation is a structural change 
needed for compensating parametric environmental un- 
certainty which is detrimental to the machine function: 
transporting particles with the maximal current [see (23) 
and the discussion around] . 

Let the environment changes as in (22), and p, can as- 
sume any value from a set M.. It resides in each of its 
states with a fixed /i for a sufficiently long time so that 
F and S do have enough time to relax to the stationary 
probabilities p" and pi\a- We require that the machine 
functions optimally, Jhiji) = •/£; ^^ each environment: 
the probability p" of the structural state with AE" = ft 
[see (21, 24)] is maximized for each }1 G Ai, i.e. the prob- 
abilities of all other states are suppressed in the sense dis- 
cussed below. The choice between the optimal structural 
states is done autonomously: after p, changes from one 
value to another (or goes back to its older value), F + S 
relax to a new stationary regime, where the structural 
state with the largest current dominates. Recall that S 
does not feel the parameter p directly, because it does 
not interact directly with the particle reservoirs; it feels 
p only indirectly due to interaction with F. Hence the 
changes of S are driven by F. 



B. Continuum limit 

We assume that 7M is a finite real interval. There 
should be a correspondence between {S}^^]^ and envi- 
ronmental states; thus to have adaptation for this case 
we need to make a a continuous variable x and to take 
in (33-35) the continuum limit: i^ ^ oo, 

^.+1 _ ^a ^ ^^(^ + ^) - E,{x) = ^d,E,{x), (36) 

6"^"+i = 6(x, ^+^) = b{x) + ^dyh{x, y)\y=,, (37) 

where b{x) — b{x, x) and where the continuous variable x 
changes in an interval: x £ [Li, L2]. Thus the stationary 
probability of S (which is now a probability density due 



calling its relaxation process adaptation amounts to trivialities. 
But the things are not anymore trivial if we take into account 
that each relaxation is accompanied by a change of some other 
quantity, which is hidden in the phenomenological description, 
but within a deeper description corresponds, e.g. to the energy 
of the reservoir that couples to the relaxator and ensures its spe- 
cific behavior. The behavior of this quantity already deserves to 
be analyzed from the viewpoint of adaptation. 
The "how" question is also an important one. In this context, 
one should distinguish robustness from adaptation. The first 
concept includes stability mechanisms that do not lead to struc- 
tural modifications in the system. These mechanisms are more 
like shielding the system from external perturbations. The prob- 
lem of combining robustness with efficiency was studied recently 
in transport models of cell biophysics [24]. 



to the continuum limit) depends on three functions Ei (x) 
and b{x). These functions look arbitrary, but we show 
below that they are fixed from the adaptation condition. 
In the continuum limit the sums over 7 in (35) can 
be replaced by integrals. As shown in Appendix A, the 
stationary probability of S then reads from (35) [A'{x) = 

a.A(x)] 

p(x) =cxp[-*(a;)]/Z, (38) 

^'{x) = 9 fix) + {9- l)0(x)[/'(x) - E[{x)l (39) 

/(x)^-ln[e-^i(=^)+e-^^(^)+^], (40) 

0(x) = [gbix) _^^^b{x)+f.-E2ix)+E^ix) ^^^~l^ (4^) 

where Z in (38) is deduced from J, ^ dxp{x) = 1. In (40), 
f{x) is the free energy of F calculated at a fixed x; see 
(17) and [23]. The second term in (39) is due to different 
temperatures of S and F {9 ^ 1) and the dependence of 
the times-scales of S on F {b{x) ^ 0); see (29, 34). 

For adaptation it is necessary that for any p G Ai, p{x) 
in (38) has a unique and sharp maximum aX x ~ x{p) 
with [cf. (21, 24)]: 



AE{x{p)) = E2{x{p)) - Ei{x{p)) = p. 



(42) 



Then x{p) will be the most probable value of x. Two 
conditions for x{p) to be a local maximum of p{x) 
are ^'{x(p)) = and *"(x(/2)) > 0. Working out 
^'(x(/2)) = and using (42) we get 



e''(*(^» - 1 _ 6 E[{x(p)) + El^ixip)) 



gh(i(^)) _|_ I 



1 



E',{x{p))-E[{x{p))- 



(43) 



This relation is supposed to hold for all p E Ai, and since 
functions Ei,2{x) and b{x) do not depend on p — because, 
in particular, S does not interact with the particle reser- 
voir — (43) holds in a dense set of x. Once Ei{x), E2{x) 
and b{x) are assumed to be smooth functions of x, (43) 
will hold for all x E [Li,L2] ^; it defines a functional 
relation between Ei{x), E2{x) and b{x): 



eK^)_l e E[{x)+E'2{x) 



(44) 



eHx) _^i i_Q E^{x) -E[{x)' 
We work now out 'ii"{x) using (44) and put there (42): 

[AE'{x{p))]^ ., 



^"{x{p)) = 
+ (l-0) 



^b(x{ti)) 



1 



gb(x(tj.}) _|_ I 



(45) 



^ The precise statement is that a mcromorphic (analytic up to 
isolated poles) function can have only isolated zeros; otherwise 
it is equal to zero [40] . If we assume that the difference between 
two sides of (44) is meromorphic, it is zero on a dense set due to 
(43) and hence is zero everywhere. 



We need 6 < 0, i.e. a negative-temperature state of S; 
otherwise x{fi) is a minimum of p(x{p,)), not maximum 
^°. We now set b{x) = 0, because then any 9 < wih 
sufhce for adaptation; otherwise, 9 has to be smaller than 
certain negative value. Eq. (44) under b{x) — implies 
E[{x) ~ —E2{x). We shall see in section V that setting 
b{x) ~ appears to be optimal from the viewpoint of re- 
ducing fluctuations of x around its most probable value. 
Recall from (34, 37) the physical meaning of the condi- 
tion b{x) = 0: the time-scale of S does not depend on 
the state of F. 

We conclude that the adaptation requires a negative 
temperature 6* < for S. This condition holds as well 
when the environmental chemical potentials (and hence 
S) assume a finite number of values; see Appendix D. 
After bringing an explicit example of adaptation, we con- 
tinue with clarifying the generality of the 9 < condition 
(section IV D) and its physical meaning (section IV E). 



For b{x) = we get from (38-41) that *(a;) := 9f{x), 
where f{x) is the free energy of F calculated for a fixed 
slow variable x. The joint probability of F + S is then 



p^{x) cx e^^'^^Vilx, 



(48) 



where Pi\^ (as a function of i for a fixed x) has a Gibb- 
sian form; see (17, 21) and recall that in the optimal 
regime tl = tr. Eq. (48) for Pi{x) implies the time-scale 
separated two-temperature system, which admits a con- 
sistent thermodynamic description despite the fact that 
the temperatures of S and F can be very different [23]. 
For such a system, the fact that the adaptation requires 
6* < is deduced from the generalized Le Chatelier princi- 
ple: 9 < is necessary that perturbations of the chemical 
potential do not amplify in time; see Appendix B. 



E. Negative temperature states 



C. An example 

Let us at this point stop the general reasoning and 
bring an explicit example of above construction (38-45): 
Ei{x) = aiX for i = 1,2 is linear function of x with 
its slope Oj dependent on i. For a fixed i, Ei{x) as a 
function of a; is a potential energy. This potential energy 
is not confining, but it is not a problem, since we anyhow 
restricted the allowed range of a;: a; e [Li,L2]- Now 
requiring ax = —02 leads to b{x) = 0; see (38-41, 44). 
The stationary probability of S reads from (38-41) 



p{x) (X (cosh[a2a; ]) '*'. 



(46) 



The most probable value of x is 2^, condition (42) is 
ensured, and the interval of allowed values of p, is 



Al = 2a2 X [Li,i2]- 



(47) 



Note that the range M. of p, over which the adaptation 
occurs is finite and enters into the hardwire of S+F as an 
apriori knowledge on the environment. M is finite due 
to the fact that S interacts with a negative-temperature 
bath: if the range of x is not finite, such an interaction 
will lead to instability; see section IV E and [27-29]. 



D. Le Chatelier principle 



Let us discuss in more detail the features of negative- 
temperature states. 

- They maximize (not minimize) the average energy 
for a fixed entropy [27-29]. 

- They thus store energy: a cyclic external field can 
extract work from them [27, 28]. Such states are au- 
tonomous sources of work [29] ^^. 

- To support 6* < 0, S needs to have a bounded phase- 
space [27-29], as already assumed ^^. 

- Negative temperatures are higher than positive ones, 
since heat fiows from lower inverse temperature to the 
larger one [27-29] . This implies that the energy stored in 
S due to 9 <0 will constantly flow to the bath of F with 
the rate proportional to e [see (31] tending to dissipate 
the stored energy. This current is calculated in Appendix 
D 2. Due to assumed e ^ 1, it is smaller than the particle 
current Jl in (16). 

- One should distinguish between population inversion 
(there are at least two energy levels such that the higher 
energy level is more populated than the lower one) and a 
negative-temperature state of a many- level system, where 
in every pair of energy levels the higher energy level is 
more populated. These two notions are equivalent for 
a two level system, but generally they are different. A 
pair of energy levels with population inversion suffices for 
(necessarily imperfect) adaptation, if one can restrict S 
to those levels, i.e. neglect transitions for higher energy 



Here we argue that the necessary condition 9 < Q for 
the adaptation is more general that the above derivation 
may suggest. 



^'^ Note that 'J'(x) can have other minima and maxima. Our concern 
here is the specific minima of ^(x) — and maxima of p(x) — that 
are given by (42). This condition is necessary for the adaptation. 



^^ Note the difference: a system with a positive temperature can 
still produce work (during cyclic action of external field) , if it is 
attached to a thermal bath at a different temperature; cf. Foot- 
note 13. In contrast, a negative-temperature system can pro- 
duce work autonomously, without external environment. Here 
the cyclic condition means that the source of work couples and 
and then decouples from the system. 

^•^ Hence the kinetic energy (which is non-negative and can be ar- 
bitrary large) cannot be a part of those degrees of freedom that 
support a negative temperature. 



levels; see Appendix D. However, in that case the adapta- 
tion is necessarily imperfect, i.e. there is no limit, where 
the probability of the undesired structural states can be 
made arbitrary small. We focussed in section IV B on 
the negative-temperature state of a many-level system, 
because it does have a limit, where the adaptation can 
be made perfect; see section V below. 

- States with population inversion can be prepared by 
pumping the system to higher energy levels; see [28] for 
an extensive review of these methods. This includes not 
only lasers and masers, but also macroscopic magnetic 
moments, rotators, dipoles etc [28]. It is also possible to 
prepare a population inversion via two weakly coupled 
systems held at different positive temperatures [29] ^^. 

- Many-body states with a negative temperature are 
well known and were experimentally realized for discrete 
and localized degrees of freedom such as the spin of nuclei 
or atoms [28, 41]. One scenario of their realization and 
observation is a microcanonic state of a many-spin sys- 
tem in the regime, where the number of available states 
decreases with increasing the energy [41, 42]. Negative- 
temperature states were also experimentally realized for 
motional states of cold atoms [43], where the energy 
spectrum of the negative-temperature carrying degrees 
of freedom is continuous, but bounded (similar to our 
example in section IV B). 

- There are complex materials held in metastable 
states that — without being properly negative- 
temperature — behave like negative-temperature states 
with respect to external variations [30]. 

- Living organisms are capable of being autonomous 
sources of work (the work is done, e.g. for preventing 
their relaxation to equilibrium). Hence they store energy 
at least via population inversion ^^. This fact is well un- 
derstood in biological thermodynamics [31-35] and was 
employed in a definition of life [34] ; see section VII B for 
more details. 



V. FLUCTUATIONS 

Wc return to the stationary probability density p{x) of 
S; see (38). Assume that the maximum x(p,) of p{x) is 
unique. There are local fluctuations around that maxi- 
mum, since p{x) is not a delta-function centered at x{fi). 
Thus there are intrinsic fluctuations of x, and thus of 
AE{x), even for an environment with a fixed p,. For 
X ~ x(/l), (38) amounts to 



p{x) oc exp[--\-^"{x{fl))\{x{p) - xf] 



(49) 



the standard deviation of x is a^ ~ l/y^\'^"{x)\. We 
need to consider fluctuations of AE{x), since F feels x 
only via AE{x). If ax is small, the standard deviation of 
AE{x) around p is 



(50) 



a=\AE'{x)\<Jx = 2/^\e\ 



where we used (45); recall that we set b{x) — 0, this is 
also optimal from the viewpoint of reducing a. 

Now the only possibility of cr ^ is to take 6* — t- — oo, 
i.e. a vanishing entropy of S due to a large amount of 
stored energy. There are two general methods of fighting 
against fiuctuations: reducing the environmental noise 
as such (e.g. cooling the environment), or putting the 
system under a strong confining potential. The second 
method does not work in our situation, since it appears 
that the strong potential cannot be adaptive. 

Thus, the adaptation is generally imperfect, because 
even at a fixed environment (for a fixed p) , AE will fluc- 
tuate on characteristic times of S. The imperfect adapta- 
tion is useful only if the environmental changes are wide 
enough; otherwise the same machine with a fixed optimal 
structure tuned to the average environment will have a 
larger average current. This point is worked out in Ap- 
pendix C. 



^^ Consider two very weakly coupled two-level systems with en- 
ergy level (0, ei > 0) and (0, 62 > 0) held at tempera- 
tures Ti = -^ and T2 = -J-, respectively. The joint sys- 
tem has four energy levels (0, ei,e2,ei -|- €2) with probabilities 



oc {l,e"''i'i,f 



-/92e2 



^e pi<!i P2«2)^ respectively. The levels ei 



and 62 of the joint system display population inversionif(ei 
e2)(e^'^i''i — e~P2<!2) > g if jn addition ei and £2 are far from 
zero and close to each other, the energy levels ei and 62 of the 
joint system can be regarded as decoupled from the rest of the 
spectrum (for certain times). 
^'^ In living organisms the population inverted states is transferred 
from one place to another via ATP [13]. It is an important and 
open question whether also in ATP the energy is stored via popu- 
lation inversion. Alternatively, the ability ATP to produce work 
may be due to different chemical potentials between ATP and 
its environment, which means that ATP is not completely au- 
tonomous, it needs an environment to produce work. McClarc 
argued that the relevant times of the ATP work-delivery pro- 
cess are such that the coupling with the environment can be 
neglected; hence, according to [32], the energy is stored in ATP 
via population inversion. 



VI. ALTERNATIVE SET-UPS OF 
ADAPTATION 

To get the proper perspective on the obtained results 
we briefly mention certain extensions of the basic set-up. 

Above we required that the set M of environmental 
values of p is dense. The same limitations (negative tem- 
perature and structural fiuctuations) are obtained when 
the environment assumes a finite number of values; see 
Appendix D. But here fiuctuations are finite even for 
6 —)■ --00. 

Another set-up is to allow S to interact directly with 
the uncertain environment [sec Appendix E]. This is 
done naturally assuming that different states F^S" have 
different particle numbers TV". The above limitations 
are then absent: even the two-state S (in face of a bi- 
nary environment) can ensure perfect adaptation in the 
isothermal situation 9 = 1. In this set-up S becomes 
an external sensor for the machine. (This is sometimes 
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called feed-forward scenario of adaptation in contrast 
to the previous scenario that works via feedback to F: 
environment ^ F ^ S ^ F.) Hence no stored en- 
ergy is needed and no fluctuations at the constant en- 
vironments are necessarily present. (Fluctuations may 
still be present due to various non-idealities.) The draw- 
back of this set-up is that it demands a specific structure- 
environment interaction that has to designed anew for ev- 
ery new environment. Put differently, the above thermo- 
dynamic costs are necessary for ensuring the autonomous 
character of the adaptation. 

Yet a different set-up is to relax the optimality con- 
dition demanding instead the stabilization (constancy) 
of the current on the face of environmental changes at 
some sub-optimal value; the above limitations are then 
also weakened. The conceptual problem with relaxing 
the optimality condition is that then the very adapta- 
tion may easily become useless: who wants to keep a 
machine that does not function well in any environment? 
It is clear however that the practical examples of adap- 
tation will be rather of this, non-optimal type, and an 
important chapter of the future research on the physics 
of adaptation should perhaps focus on understanding the 
trade-offs between optimality and thermodynamic costs. 



VII. DISCUSSION 
A. Summary 

Thus our adaptive machine consists of two parts F and 
S (functional and structural degrees of freedom, respec- 
tively) and works as follows; see Figs. 1, 2, 3. If the ex- 
ternal chemical potentials are constant in time — and pro- 
vided that the fluctuations of S around its most probable 
value can be neglected — the functional degree of freedom 
F transports particles with the maximal current (speed) 
from the higher chemical potential to the lower one. 

Let now the chemical potentials change. After they 
settle in new values, F will transport particles with much 
smaller current (speed) than it is allowed under new val- 
ues of the chemical potentials [see section II, 2]. F will 
then act on S, and after a much longer time (much longer 
because S is much slower than F) S will feedback on F 
making the current again maximal under the new values 
of the chemical potentials. For this adaptation process 
to happen it is necessary that the structural degree of 
freedom S has a sizable amount of stored energy, i.e. its 
temperature is negative 6 < 0. A negative 6 is needed for 
ensuring that perturbations induced by changing chem- 
ical potentials are not amplified in time. If, simultane- 
ously, 1^1 is sufficiently large, the fluctuations of S around 
its most probable value are negligible. 

However, because 9 is necessarily finite, there are in- 
trinsic fluctuations of S that sometimes force the current 
to deviate from its maximal value even under a constant 
environment. The origin of these fluctuations inherently 
relates to the adaptation: it is impossible to prevent fluc- 



tuations by conflning S via an external potential, since 
the latter will spoil the adaptation. 

On very long times — which are conceivable, but not 
directly seen on this model — the second law will force 
9 to increase due to heat exchange with the baths of F 
[27]. Hence S will make more errors (fluctuate stronger) 
even in the constant environment; see (50). For even a 
larger 9 having a variable structure S may be useless: 
fluctuations of S are then so strong that in the sense of 
the average current it is better to have a constant (non- 
variable) structure [see Appendix C]. Thus, adaptation 
is not always useful. 



B. Relations w^ith thermodynamic theory of aging 

The functioning of the adaptive machine resembles cer- 
tain features of living organisms, in particular, the pro- 
cess of aging (senescence) that is generally defined as pro- 
gressive loss of stability of an organism that increases the 
probability of its failure and that arises out of the normal 
functioning of the organism [36] . Aging is a complex phe- 
nomenon with many inter-related mechanisms at play; 
various theories of aging emphasize different mechanisms 
[36]. We shall focus on analogies between the function- 
ing of the adaptive machine and the thermodynamic the- 
ory of aging, whose different aspects were uncovered in 
[31, 35, 36]. The theory is one under development, but 
it is already recognized in gerontology [35, 36]. Since it 
is not well-known in statistical physics, we shall briefly 
review the main postulates of this theory and then out- 
line their similarities with the adaptive machine studied 
above. 



1. Postulates of the thermodynamic theory of aging 

1. Living organisms are in a non-equilibrium state that 
is characterized by a certain amount of stored energy (i.e 
energy related to population inversion) [31, 35]; see sec- 
tion IV E. This stored energy is needed for performing 
various function, e.g. the organism needs it for prevent- 
ing (doing work against) its own equilibration [31]. 

2. For simplicity we shall visualize this stored energy 
as a negative-temperature reservoir at temperature —\9\ 
(though in reality it will most likely has more complex 
forms not described by a single negative temperature). 
The reason why the non-equilibrium state is character- 
ized by a negative temperature (or population inversion, 
but not just an excess free energy with respect of a given 
thermal environment) was already explained in section 
IV E: the organism should be capable to perform work 
autonomously. 

3. The stored energy is inborn, because the organism 
cannot obtain it only from digesting (non-equilibrium) 
food [31, 35]. Indeed, digestion is a complex process that 
itself needs investment of work [31, 35]. It is assumed 
that some amount of stored energy is contained already 
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in the seed (for plants) or in the zygote (for mammals) 
[31, 35]. 15 

4. During the life of the organism, \6\ monotonically 
decreases, due to various functions performed by the or- 
ganism (and due to inevitable coupling of the negative- 
temperature bath to positive-temperature baths) [31, 35]. 
Aging refers to state, where [^l is so small that the organ- 
ism cannot perform adequately its functions. In particu- 
lar, controlling systems of the organism get progressively 
less stable and less efficient in responding to signals from 
functional degrees of freedom. This point is emphasized 
in the ontogenetic theory of aging [37] . 

5. The free energy provided by food does not increase 
\9\, it is only used for preventing its fast decrease and 
for extending the size of the negative-temperature bath, 
i.e. the amount of stored energy can increase, with its 
temperature 9 < still decreasing by its absolute value 
[31, 35]. The situation where the amount of the stored en- 
ergy increases corresponds to the growth of the organism 
[31, 35]. Hence a mature organism (the one that does not 
grow anymore) has a limited amount of resource available 
for its functional tasks; this point is fundamental for cer- 
tain other theories of aging, notably for the disposable 
soma theory [36]. 



2. Relations with ttie adaptive mactiine 

The above points 1 is basic for the adaptive machine: 
the stored energy in the structural degree of freedom S 
is needed for the adaptive functioning of the machine 
(stability of the maximal current under environmental 
changes). Also, the point 2 is there, since it is only 
the simplest models of S that require a single and well- 
defined negative temperature. 

The gradual decay of |6'| and the ensuing instability for 
the adaptive machine — to an extent that having a com- 
plex, adaptive structure is detrimental — resembles the 
aging process; see our discussion in section VII A and 
points 3 and 4 above. Note as well the relation with 
the ontogenetic theory of aging that stresses progressive 
losses in controlling systems of the organism. 

For the adaptive machine the external non-equilibrium 
(ml 7^ /^r) environment cannot be used to support the 
negative-temperature state of S, we needed to assume a 
separate negative-temperature thermal bath for S (points 
3 and 4 above). In our situation, the size of the bath of 
S is large but fixed, which correponds to a mature (not 
growing) organism; see 5 above. 



^^ One of the most cited points on non-equilibrium character of 
living organisms belongs to Schrodinger who opined that organ- 
isms feed on negative entropy [38] . This point is questionable for 
two reasons; first, it does not recognize the inborn character of 
the non-equilibrium (stored energy) , second it does not take into 
account that food contains mostly energy, not negative entropy 
[39]. 



Recall that above analogies came out from combining 
a statistical thermodynamics model with the stability of 
the maximal current under environmental changes. Note 
that though the above conditions for adaptation are ob- 
tained for a particular model, we argued that they hold 
more generally, and are based on the quasi-equilibrium 
transport and the Le Chatelier principle. 
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Appendix A: Derivation of Eqs. (38—41) 

Before applying (33) to (35) we change in (33) a -^ 
7 -I- 1 and j3 -^ ^. Next, applying (36, 37) we get for 
the first factor in (33) within the first order of the small 
parameter -^i 



9(EJ-E1- 



e K 



E[{x) 



(Al) 



where Ei{x) = Ej and E[{x) — dxEi{x). Likewise, we 
obtain for other factors in (33) 



1 



ati-^E-l 



1 + ef'-^E"' 

l + gb^ + ^-^ + KAB 



exp 



AE'{x) 
K 



„p-AB(a;) 



1 _|_ gfi-AE{x) 



2 I gh-'+i.-' + |(AE-' + i-A£;-')-A£;-'+i+Ai 

{9 - l)AE'{x) e''(^)+^-'^-^(^) 



exp 



K 



I _|_ pb(x)+il-AE(x) 



,(A2) 



(A3) 



where AE = E2{x) - Ei{x) and b{x) = b''+^-^ + 0{-^); 
see (36, 37). Combining (A1-A3) into (35) and changing 
there -^ E7 ^ /da; we get (38-41). 



Appendix B: Le Chatelier principle 

Here we shall derive the Le Chatelier principle for 
perturbations of the chemical potential. In contrast to 
known derivations of the principle that are purely ther- 
modynamical, the present derivation stays within sta- 
tistical mechanics. We want to relate the principle to 
conditions required for adaptation. 

Though the Le Chatelier principle is widely known and 
frequently applied outside of physics (see [45] for fur- 
ther references on such inter-disciplinary applications), 
already its thermodynamic derivation contains several 
subtle points; see [44] for a review. The statistical me- 
chanic derivation of the Le Chatelier principle for pertur- 
bations of an intensive variable (chemical potential) is to 
our knowledge presented for the first time. For pertur- 
bations of extensive variables, the principle was recently 
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discussed from the viewpoint of kinetics (non-equilibrium 
statistical mechanics) [45] . 

We shall work within a set-up close to that in the main 
text. There are two interacting systems S and F. Now S 
is described by coordinate (continuous variable) x, while 
F can be in discrete states i — l,...,n. Each such state 
has energy Ei{x) and carries Ni particles; recall our dis- 
cussion in section II A. The coupling between S and F is 
due to dependence of Ei{x) on x. 

F couples with a reservoir at temperature 1 and chem- 
ical potential /2. S couples with an energy reservoir at 
inverse temperature 9. S is much slower than F. We 
assume that the conditions for the two-temperature adi- 
abatic quasi-equilibrium hold — see the discussion after 
(45) and [23] — which means that the stationary proba- 
bility p{x) of S and the conditional stationary probability 
pi\x of F read 

p-Ei(x)+p.N, 

Pa. = r., .. , Z(.x) = ^.e-^-(-)+^^-, (Bl) 



Z{x) 



p(a;)cxe'""^(^). 



(B2) 



Thus, the conditional stationary probability of F has the 
Gibbsian form with the chemical potential p and tem- 
perature 1, while the stationary probability p{x) of S is 
given by the Gibbs distribution (at inverse temperature 
6) with the free energy — InZ(x) of F calculated at a 
fixed value of x. 

Recall that having two time-scales (slow and fast) is the 
basic premise of the Le Chatelier principle that shows up 
in all its formulations; see [44, 45]. If the two tempera- 
tures are equal, = 1, (B2) reverts to the usual Gibbs 
density. Denote: 



{N). 



dxp{x){N)^. 



(B3) 

(B4) 



Since S is slow and F is fast, {N)x characterizes the av- 
erage particle number in F for intermediate times, when 
the state x of S is fixed. {N)^ is the average of {N)x 
over all states of S; it naturally characterizes the average 
particle number in F for long times, where fiuctuations 
of S are essential. 
Eqs. (B1-B3) imply 



dp {N)x - df,{N)x = 6 {N)l - (TV), 



(B5) 



For > 0, (B5) is non-negative. This is statement of 
the Le Chatelier principle for perturbations of intensive 
variables (chemical potential): they are amplified in time 
[44]. Indeed, dfi{N)x in (B5) is the response of S + F to 
a perturbation of p that is much faster than S, but much 
slower than F. dp, {N)x is the response of S -I- F to the 
same perturbation, but on much longer times so that S -1- 
F has thermalizcd at the perturbed value. Recall that for 
perturbations of extensive variables the statement of the 



Le Chatelier principle is just the opposite: perturbations 
are suppressed (not amplified) in time [44, 45]. 

Now recall the set-up of section IV B and assume that 
p{x) « S{x — x{p)) is a well localized around its average 
that is a necessary condition of perfect adaptation. We 
now obtain for > 



(55) = / da; 



dp{x) 
dp 



{N)x = dpx{^j) dx{N)x\x=x{ti) 



= -~dpx{p) AE'{x{p)) dp{N)x > 0, (B6) 

where took into account that {N)x depends on x 
only through AE{x) — p. Using dp_{N)x > that 
also follows from the Gibbsian property (Bl) we get 
dfix{p) AE'{x{p)) < contradicting to the condition 
AE{x) = p oi adaptation; see (42). 



Appendix C: Quantifying imperfect adaptation 

Section V shows that the fact of structural fluctuations 
is an unavoidable consequence of adaptation. We expect 
that such imperfect adaptation is not useful when the 
variance of environmental changes is small. To quantify 
this aspect, we need to specify the statistics of environ- 
mental changes. For simplicity we assume that in (22) 



ML - MR = Ml - Mr- 



(CI) 



Hence p = ^('^l + Mr) with probability V{p) is the only 
changing environmental parameter. 

Let us now consider the current (16) that is partially 
optimized: we take in (16) tl = t-r, = T, because this 
maximizes Jl with all other parameters being fixed; see 
(19). For technical simplicity we fix the parameters as in 
(24), i.e. we take T = 1. The resulting expression for Jl 
reads 



A[AE{x)-p] = 



sinh[(^L - Mr)/4] 
cosh[(A£;(a;) - p)/2] 



(C2) 



We now average (C2) over over structural and environ- 
mental noise: 



Jl = / dxdpJi^[AE{x) - p]p{x)r{p), 



(C3) 



where p{x) is the stationary state of S; see section V. 

This is to be compared with the situation when the 
structure is fixed: first AE is fit to its optimal value 
for one environment, e.g., the one at (p) = J dppV{p), 
thus AE = (p) [recall (24)]. Then the average over the 
environments is taken: 



J2 



d^lJL[{^J.)- ^A'P{^l). 



(C4) 



We expect that J2 > Ji (Ji > J2) whenever the width 
of V is sufficiently smaller (larger) than that of p; the 
adaptation is then useless (useful). 
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Let us now consider the example (46): Ei = aiX for 
i — 1,2 and x G [Li,L2], where ai = —02, 6 = and 
p{x) ex (cosh[a2a; — |-])^l''l. We assume that V{il) is 
Gaussian with the standard deviation d and mean (/2). 
We get 



Ji J da; (cosh [a;]) 
J2 " 



/dx(cosh[x])-l^l J dx exp(-|^ - lncosh[|])' 

Now J1/J2 is larger than one — and thus the adaptation 
is useful for this example — if, e.g. d = 1 and 16*1 > 4.811. 
For d — > 00, J1/J2 oc vd tends to infinity: the adapta- 
tion is always useful for sufficiently large environmental 
uncertainty. 



Appendix D: Two-state environment 

1. Adaptation 

Let the environment can be in two states £1 and £2, 
where jl — (/xl + Mr)/2 assumes two different values /2i 
and fl2- These values are known, but it is not known 
which one will be realized (cf. section IV A). Then the 
minimal (necessary for adaptation) number of states for 
the structural degree of freedom S is also two, and the 
structural and environmental states match as 



£„ O S", E^-E^ = fl^, a = 1, 2, 



(Dl) 



where the second condition means that that each state 
S" provides the optimal value for E2 ~ Ei [cf. (42, 21, 
24)]. For the stationary probabilities of S we get [see (35) 
for K ^2] 



Using (D2) and (33) we obtain 



pHmi) 
pHm2) 

P^(M2) 



MEl-E-^,) 1±£! 



1 + e'>+^ 



2 l + e''+(i-f)' 
1 + e-™ 1 + e}-^ 



l + e''-(i-f)' 



where we defined [cf. (29, 34)] 

m = fli-fl2, b=(Bl^-Bf)/2. 
Eqs. (D3, D4) imply 

l-pi(M2) 1-P^{iii) 
_ 1 + cosh[m] cosh[6] + cosh[^]) 



(D2) 

-, (D3) 
-, (D4) 

(D5) 



cosh[6] + cosh[^ - m] ' 



(D6) 



For the perfect adaptation it is necessary to have both 
P^(fi2) and p^{jj.i) going to zero (cf. section IV A). Here 



P^ifi2) andp^(/ii) are the error probabilities, they charac- 
terize deviations from the optimal behavior due to struc- 
tural noise. If they both go to zero, then for £ — £1 
{£ = £2) one can neglect transitions to S^ (S^), at least 
in the stationary regime, and the machine will function 
optimally for each environment; see (42). 

The perfect adaptation is impossible: the minimum of 
(D6) equals to 1/4 and it is reached for 



<0, 



0, |m| > 1, |6l||m| > 1. (D7) 



Indeed, < is necessary for (-D6) < 1; the remaining 
conditions in (D7) is straightforward to obtain and inter- 
pret: 6 = means that F does not alter the time-scales 
of S, while |to| 3> 1 means that the environmental values 
are sufficiently different from each other. In this context 
recall our discussion on h{x) = after (45). 
The imperfect adaptation can be defined via 



P^(M2) < P^(M2), P^(mi) < P^(Mi), 



(D8) 



i.e. the probabilities to get into "wrong" structural states 
are smaller than the probabilities to be in the "right" 
states. Eq. (D8) means that the error probabilities p^(/i2) 
and p^ifii) are both smaller than 1/2. For this it is nec- 
essary that 9 < 0; otherwise (D6) is always larger than 1. 
Hence the imperfect adaptation demands negative tem- 
peratures; cf. section IV B. 

For illustrating the imperfect adaptation let us assume 
that i) ^L —Mr stays constant [cf. (CI)]; ii) \m\ ^ 1; Hi) 
both states £1 and £2 have equal probabilities 1/2. 

Now if the environment is in £„ (a = 1, 2), S is in its 
states S^ and S^ with probabilities p^{p.a) and p^{pa), 
respectively. If S is in S", the particle current is given 
by Jl = sinh[(^L - Mr)/4], see (C2). If S is not in S", 
the particle current is J£/cosh[?Ti/2]; cf. (23). This cur- 
rent is negligible due to the above condition ii). Hence 
the current averaged over environmental and structural 
fluctuations reads 



i[pi(Ali)+p2(M2)]J£. 



(D9) 



We compare (D9) with a constant 8 = 8-'^ that is fit 
to the optimal value E\ — E\ = E2 — Ef — p,i of one 
environmental state. In that situation the average par- 
ticle current is -kJl- Requiring that ij£ is smaller than 
(D9) we get p^ (jli) + p"^ {p,2) > 1, which is ensured by the 
definition (D8) of the imperfect adaptation. 

Concrete conditions for imperfect adaptation are found 
from (D3, D4) under 6 < and (D8). We mention 
the particular case, where the error probabilities p^{jj.2) 
and p^{jli) both assume their minimal values. Consider 
[ml ^ 1 and 6 = 0, when (D8) amounts to 



El - El 



Ml -M2 



The minimal values p^(/i2) 
when \E\ -Ef + ^i^| - 
consistent with (D7). 



<W <™' 



= p^ini) — 1/3 are reached 
0. These minimal values are 
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Hence the two-state structure is able to adapt imper- 
fectly to two environmental states that are sufficiently far 
from each other. For the imperfect adaptation we need 
not only that the temperatures of S and F differ (9 7^ 1), 
but also that the temperature of S is negative (9 < 0). 



2. Energy current from S to F 

As we have seen, adaptation demands that the temper- 
ature 6' of S is different from that of F [which is defined 
to be 1] and that 9 < 0. Then there will be a current 
of energy Js->f from the bath of S to that of F. This 
energy will flow through S -I- F. We now study this en- 
ergy current in the stationary regime using (27-32). By 
definition, the energy flowing from the thermal reservoir 
of S is given as that part of the average energy ^^^ E'j'pf 
change which is driven by the reservoir [see (27)]: 






■ - ^1 Pi ] 



(Dll) 



Using the effective master equation [see (31, 32)] we 
rewrite (Dll) for the stationary situation as 



js^F = e5]. i?r[<>7-c.7° 


pf] 


(D12) 


= -2ef72ipia0ln[f^2iJ7i2] 


(D13) 


ggi(BP+Bi2) msinh 


(l-e)m 
2 




(D14) 


^12+ ^21 2 CO 


sh[f] 


' 



where m is defined in (D5). We see that Js-s-f > for 
9 < 1 confirming that the energy flows from the lower 
inverse temperature to the higher one. Thus, Js-s-f > 
means that the stored energy is lost {dissipated) in time, 
i.e. that the quality of adaptation, which was related 
to 6* < 0, tends to degrade in time. Note that due to 
the assumed adiabatic limit e — > 0, Js-s-f > is much 
smaller than the particle current Jl • The latter is due to 
the motion of F and is inversely proportional to the flrst 
power of its characteristic time. 



Appendix E: Controller directly sensing tw^o-state 
environment 

We now show that if we allow the structure S to inter- 
act directly with the uncertain environment (reservoirs), 
there is a perfect adaptation for a two-state S having the 
same temperature as F. Recall that the temperature of 
F was assumed to be 1, so from now on this is the com- 
mon temperature of S and F. For the present model such 



an interaction is set naturally if we assume that the state 
FiS" carries N^ particles, and it is switched off naturally 
if N'^ does depend on a. Instead of (29) we get 



a-t 



^■IT ■ 

2 L 



07 

ilR' 



i = l,2, (El) 



^"7 _ i(£7--Er)-ii(w7-ivr) 



k == L, R, (E2) 
where we assumed 9 — 1^ since S and F have the same 
temperature [see (6)], and we put B'^^ = for simplicity. 
The effective transition rates fi^^ and f2^^ are deflned 
as in (32). They are calculated from (El, E2) by using 
(32, 17): 



(il2 pi 



1 



,-AE^+fiAN'^ 



,(E3) 



where AS" = E^ - E^ and A7V" = N^ - N^. The 
probabilities p" of the states S" are defined via (E3, D2) 
A simpler way to obtain (E3) is to note that fj^^/fj^^ = 



J'-f' 



where /" is the free energy of the fast F for a 



fixed state S" of S: 



;-/° = ^-Ef+flN^ _^^-ES+fiN° 



(E4) 



Now the state of S does not depend on the time-scale 
separation, since due to 6* = 1 the overall state of S -I- F 
is Gibbsian. 

We now assume the uncertain binary environment de- 
fined in the previous section. To ensure that F functions 
optimally in each environment we set AE" ~ /i^ for 
a = 1,2 and AA^^ = AA^^ ^ ^ j^f ^-^^^i (24)]. Hence 
Nf — N^ = N does not depend on i. For the error prob- 
abilities we obtain from (E3) [cf. (D6)]: 



P\fii] 



2Q-(E\-El)-fiiN+ij.2-fii 
\ -I- g(A'2-A'l) 



P^(M2 



2f,El-El+fi2N 
1 _(_ g(Ai2-Mi) 



(E5) 
(E6) 



Conditions for imperfect adaptation are read-off from 
(E5, E6, D8). There are environments, where both error 
probabilities p'^{p,i) and p^{p,2) go to zero, e.g. take in 
(E5, E6): 



Ml > IM2I 



-M2, 



|Ar| = -A^>l, El =El (E7) 



Thus if S senses the environment directly, all adapta- 
tion costs disappear: it is possible for 6* = 1 (equal tem- 
peratures for S and F, no need for the stored energy), 
without malfunctions (both error probabilities can go to 
zero) and with the minimal number of states for S. 
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